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THIS ARTICLE DESCRIBES RESULTS OF THE FIRST IN A SERIES 
OF THREE LARGE-SCALE LONGITUDINAL STUDIES CONDUCTED BY THE 
U.S. EMPLOYMENT SERVICE TO INCREASE THE USEFULNESS OF THE 
GENERAL APTITUDE TEST BATTERY (GATB) FOR COUNSELING HIGH 
SCHOOL STUDENTS. THE FINAL SAMPLE CONSISTED OF 26,708 HIGH 
SCHOOL STUDENTS. ALL WERE TESTED WITH THE GATB IN 1958, AND 
THOSE IN THE LOWER HIGH SCHOOL GRADES WERE RETESTED IN THE 
12TH GRADE. RESULTS WERE AS FOLLOWS— (1) STABILITY 
COEFFICIENTS WERE HIGHEST FOR THE 11TK-GRADE SAMPLE AND 
LOWEST FOR THE NINTH-GRADE SAMPLE, (2) INTERVAL BETWEEN 
INITIAL TESTING AND RETESTING HAD VERY LITTLE RELATIONSHIP TO 
SIZE OF PRACTICE EFFECT, (3) MATURATION INCREASES WERE 
LARGEST BETWEEN THE NINTH AND 12TH GRADE AND SMALLEST BETWEEN 
THE 11TH AND 12TH GRADE, AND (4) OCCUPATIONAL APTITUDE 
PATTERN STABILITY WAS INCREASED THROUGH USE OF A "BAND" 

AROUND THE CUTTING SCORES. THIS SPEECH WAS PREPARED FOR THE 
AMERICAN PERSONNEL AND GUIDANCE ASSOCIATION CONVENTION 
(WASHINGTON, D.C., SESSION 231, APRIL 7, 1966). (JH) 
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Abstract 

This article describes results of the first in a series of three large- 
scale longitudinal studies conducted by the U. S. Employment Service to 
increase the usefulness of the GAI£ for counseling high school students. 

The final sample consisted of 26,708 high school students. All were tested 
with the GATE in 1958, and those in the lower high school grades were retested 
in the twelfth grade. Results were as follows : (l) Stability coefficients 

were highest for the eleventh grade sample and lowest for the ninth grade 
sairple, ( 2 ) Interval between initial testing and retesting had very little 
relationship to size of practice effect, (3) Maturation increases were largest 
between the ninth and twelfth grade and smallest between the eleventh and 
twelfth grade, (4) GAP stability was increased through use of a "band” around 
the cutting scores. 



Ihe General Aptitude Test Battery (GATB) has been widely used in the 
vocational counseling of high school seniors for the past 15 years. During 
the 1963-64 school year, the Employment Service tested about 340,000 seniors 
in about 10,000 schools for this purpose. Aptitude norms for lower high 
school grades were developed several years ago (Droege, i960), and the GATB 
has been used increasingly by school counselors in the educational- vocational 
counseling of students at the ninth and tenth grade levels. Several hundred 
schools have arranged to use the GATB in this way through release agreements 
between State Employment Services and State Departments of Education or 
individual schools (Culhane, 1964; Wy song, 1965). 



In the spring of 195& the U. 3. Employment Service initiated a series cf 
three large-scale longitudinal studies to increase the usefulness of the 
GATB as a tool for counseling high Sv’.hool students. The overall design involved 
testing students in the ninth, tenth and eleventh grades and retesting them 



*Robert C. Droege is a Research Psychologist in the Branch of Occupational 
Test Development, U. S. Employment Service, Washington, D. C. 
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in the twelfth grade. (There was also provision for testing a control group 
of students in the twelfth grade at the time students in the lower grades were 
tested initially. ) The first of the three studies was concerned \dth obtaining 
longitudinal data on effects of maturation on aptitude scores. The purpose 
of the second study vt.s to obtain data on validity of GATB aptitude scores 
for predicting academic success in high school. The purpose of the third 
study was to determine the validity of GATB aptitude scores and Occupational 
Aptitude Patterns for predicting success in college and occupations two years 
after high school. Figure 1 shows the data collection schedule for the three- 
study series. The data for all three studies have been collected and the data 
for the maturation study have been analyzed. 

The primary purpose of the GATB longitudinal maturation study was to 
investigate effects of the maturation or growth process on aptitude scores. 
Specific aspects of maturation studied were (l) stability of aptitude 
measurement! in lower high school grades and (2) average aptitude score increases 
in. high school attributable to effects of maturation or growth. Although 
maturation was the primary focus in the study, it was possible to obtain data 
also on effects of practice (previous exposure to GATB testing) on retest 
aptitude scores as a by-product of the data analysis. 

The importance of stability of aptitude measurement in lower high school 
grades, and the possible effects of maturation on stability, should be 
emphasized. Aptitude tests cannot be used with confidence unless there is 
evidence that they have substantial stability of measurement over a period 
of time. 

A recent study has shown that the aptitudes of the GATB do have good 
stability for adult groups when the interval between first and second adminis- 
tration of the test is as long as three years (Droege, in press). But other 
studies have shown that there may be differences in the rate of progress of 
the maturation process for individuals who may not have reached full aptitude 
maturity. Thus, a question arises as to whether individual differences in 
rate of maturation have a serious detrimental effect on stability of aptitude 
measurement in lower high school grades. If so, the test scores of the younger 
high school students could not be expected to provide stable indications of 
occupational and educational potentialities. It would follow that use of 
aptitude tests for long-range counseling of students in lower high school 
grades cou ld not be justified. Some data on GATB aptitude stability for 
high school samples are available from earlier studies (Droege) but 
they do not represent a systematic effort bo obtain data on comparable samples 
of boys and girls and comparable samples of ninth, tenth, and eleventh graders. 



^ ihe term "stability of measurement" is used here to refer to the relation- 
ship between initial test scores and retest scores for a specified group of 
individuals. 
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PROCEDUHE 



The experimental design included both test-retest and independent -sample 
approaches for investigating effects of maturation on test scores. As pointed 
out before, maturation may affect both stability of measurement and level of 
aptitude score. An indication of aptitude stability over the period between 
initial testing in a lower high school grade and retesting in the twelith 
grade ns provided by the prjduct -moment correlation between initial test 
scores and retest scores. But an estimate of the average increase in score 
level attributable to effects of maturation cannot be made from a simple 
comparison of initial test and retest data. Some of this increase in scores 
upon retesting results from the initial testing experiences (practice effect;. 
Estimation of the portion of the increase attributable to practice effect and 
the portion attributable to maturation is possible through use of an independent 
control sample. In this study the control sample was the sample jested in the 
twelfth grade at the time the experimental samples were tested initially m 
lover high school grades. The control and experimental samples were -ompa ratio 
in the sense that they all included only "survivors" to the same point in the 
twelfth grade. Comparisons of scores of the control sample with thv ,velrth 
wade retest scores of the samples tested initially in lower hign u .icol 
grades provides a basis for estimating average practice effects. 
of scores of the control sample with the initial scores of samples testeid m 
lower high school grades provides a basis for estimating average maturation effects. 
!Jhen the average effects of practice and maturation are estimated in this way^ 
their sura is equal to the difference between initial test mean score and re- 
test mean score for the experimental tamples. 

Nineteen State Employment Services, in cooperation with 168 schools, 
participated in the data collection. The samples were obtained from schools 
where it was possible to test students at all grade levels and to retest 
the ninth, tenth, and eleventh graders in the twelfth grade. In most instances, 
substantially all students in all four high school Grade levels of tne 
participating school were tested, v/hen this was not possible, a sample 
was selected for testing. 

The initial testing was dene during the period February 1 through 
April ^0, 1958 . The twelftn graders were tested with Form A of the B-1002 
edition of the GATE. The eleventh, tenth, and ninth graders were -ested 
with Form 8 of B-1002. They were retested as twelfth graders with Form A 
during the period February 1 through April ?-■ in the years 195 9 j I960 and 1961, 
respectively. 

Of the 15,9/; students initially tented for the study, 26, 70* were 
included in the final samples. Those neu in the final samples were excluded 
for a variety of reasons, including dropout or transfer to other schools 
during the period between initial testing, and retesting, incomplete data 
available, and absent rvm school on. retesting date. Table 1 shows the 
number of cases in the final samples by gra .e, sex and sice .>f school. 
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Tatle 2 shows the mean and standard deviation of years of age at initial 
testing for the samples. The boys were slightly older and slightly more 
variable j.n age than girls at each grade level. Otherwise, there were no 
irregularities in the age data for the samples. 

TABLE 2 

MEAN AND STANDARD DEVIATION OF YEARS OF AGE AT INITIAL TESTING 



Boys Girls 



Grade 


N 


M 


S.D. 


N 


M 


S.D 


9 


3396 


14.9 


.6 


3680 


14.8 


.5 


10 


3348 


15.9 


•6 


3»+91 


15.8 


,5 


11 


3229 


16.9 


.6 


3395 


]£.8 


.5 


12 


3028 


17.9 


.6 


3139 


17.8 


.5 



RESULTS 



Table 3 shows the GATB aptitude means and standard deviations for the 
four samples* This table contains basic information used to generate other 
tables and graphs to be introduced. Before proceeding further, however, 
the folloi/ing points about Table 3 should be noted; 

1. The aptitude mean scores for the twelfth grf.de sample tended to 
be somewhat higher than 100, the mean for the GATB General Working 
Population sample. The standard deviations were lower than 20, 
the standard deviation for the GATB General r- king Population 
sample. 

2. Aptitudes with the largest male-female differences in mean scores 
were Spatial Aptitude (boys averaged higher), Form Perception, 
Clerical Perception, Motor Coordination, and Finger Dexterity 
(girls averaged higher). The findings are consistent, applying 
equally ;rell to the results for the ninth, tenth, eleventh and 
twelfth grade samples. These findings confirm results of previous 
research on sG : differences in aptitude scores for high school 
samples (U. S. Department of Labor, 1962) and are similar to 
results obtained for a sample of adults with a wide age range 
(Droege, Craabert, and Konkin, 1963). 

3. Retest mean scores were higher than ini* ial test means, indicating 
the operation of effects of maturation and/or practice. 
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Average I ncreases in Scores Attributable to Maturation and Practice Effects 

Tfcble 4 shows the differences between initial ;est means and retest 
means for each of the experimental samples. The gross Increases, all statist!* 
ca ^y significant, are a function of maturation and practice effects. The 
technique used for dividing the total increase in mean score into that portion 
due to effects of practice and that due to effects of maturation is described 
under Procedure. Since the differences in results for boys and girls were 
not large, they were averaged and graphs were prepared showing the average 
effects of practice (Figure 2) and maturation (Figure 3) for the three 
experimental samples. 

The graphs in Figure 2 were based on differences between the mean scores 
of the twelfth grade sample and tte twelfth grade retest mean scores of the 
ninth, tenth, and eleventh grade samples. The graphs show that the average 
effects of practice (or exposure to the initial GATB testing) were very 
similar for the three experimental samples. ^J&ength of time between initial 
testing and retesting had little relationship to size of practice effect. 

Finger Dexterity and Manual Dexterity showed the largest and Verbal Aptitude 
and Numerical Aptitude showed the smallest increases in scores attributable 
to practice. These findings apply equally to the results for the ninth, 
tenth, and eleventh grade samples and, as inspection of Table 4 will indicate, 
equally to results for boys and girls. 

Bie graphs in Figure 3 were based on differences between mean scores 
of the twelfth grade sample and the meannof the initial scores of the ninth, 
tenth, and eleventh grade samples. The pattern in Figure 3 (maturation) 
is quite different from that in Figure 2 (practice). Although the shapes 
of the profiles for the three experimental samples tend to be parallel in 
both figures, the profile levels vary quite noticeably in Figure 3j a 
reflection of cumulative effects of maturation processes from the ninth to 
the twelfth grade. Thus , (maturation increases for all aptitudes were largest 
between the n^nth and twelfth grades and smallest between the eleventh and 
twelfth gradey where the sizes of the increase due to maturation were quite 
small. Average maturation effects from the ninth to twelfth and from the 
tenth to twelfth grades were largest for Motor Coordination and smallest 
for Spatial Aptitude. As in the case of practice effects, inspection of 
Table 4 shows that the results relating to average effects of maturation 
were quite similar for boys and girls. 
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MEAN SCORE INCREASES FOR THE APTITUDES OF THE GATB 
(See Table 1 for Number of Cases) 
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Stability of Aptitudes and QAP 1 3 

Aptitude stability coefficients are shown in Table 5 and in Figure 4 . 
These stability coefficients are the product-moment correlations between 
initial test scores in lower high school grades and retest scores in the 
twelfth grade. The following points should be noted: 

1 , The profiles of stability coefficients for the experimented samples 
are parallel, The coefficients for all nine aptitudes were highest 
for the eleventh grade sample and lowest for the ninth grade sample. 
Medians and ranges of stability coefficients were as follows: 



2 , The aptitudes with the highest stability coefficients were Intelligence 
Verbal Aptitude and Numerical Aptitude, The aptitudes with the lowest 
stability coefficients were Form Perception and Finger Dexterity, 

These findings were consistent, applying equally well to the results 
for the ninth, tenth and eleventh grade samples, 

3 , The boy-girl differences in stability coefficients were quite small, 
and the findings in 1 and 2 above, which were based on averaged 
stability coefficients, applied quite well to the data for boys and 
girls separately. 

Twenty years of occupational research with the GATB have led to the 
development of validated Occupational Aptitude Pattern (QAP) norms for families 
of occupations requiring similar abilities (U, S, Department of Labor, 1962), 

The QAP norms consist of cutting scores for three significant aptitudes 
required by the occupations in the family. Thirty-six QAP 1 s have been 
established and they cover about 850 occupations. To obtain information on 
stability of QAP’ s at the ninth, tenth and eleventh grade levels, QAP norms 
were applied to the Initial and to the retest scores for each sample (after 
appropriate adjustments were made in the cutting scores for maturation and 
practice effects) and the relationships between pass-fail on initial test 
scores and pass-fail on retest scores were obtained, A summary of these 
results, in terms of the distribution of tetrachoric correlation coefficients 
between pass-fail on initial scores and pass-fail on retest scores is shown 
in Table 6, 

There was seme variation in the distributions of tetrachoric correlations 
for boys and girls, but the medians were the same for the ninth and eleventh 
grade samples and only slightly different for the tenth grade sample. 

The distributions for boys and girls together are shown graphically in Figure 5 * 
It is apparent that stability tended to be highest for the eleventh grade 



Median 



Range 



Ninth grade sample 
Tenth grade sample 
Eleventh grade sample 



759 



•5& - «o0 
.62 - ,83 
,64 - ,84 



.72 

.75 
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TABLE 5 

STABILITY COEFFICIENTS FOR IHE APTITUDES OF THE GATB 
(See Table 1 for Number of Cases) 



Aptitude 



r 9 9R r 10 10R r ll UR 



G 



V 



N 



S 



P 



Q 



K 



F 



M 



Intelligence 



Boys 


.78 


.82 


.84 


Girls 


.80 


.83 


.83 


Average 


.79 


.82 


.83 


Verbal Aptitude 


Boys 


.79 


.82 


.82 


Girls 


.79 


.81 


•82 


Average 


.79 


.81 


.82 


Numerical Aptitude 


Boys 


.78 


.80 


.83 


Girls 


.76 


.78 


.80 


Average 


.77 


.79 


.81 


Spatial Aptitude 


Boys 


.72 


.76 


.75 


Girls 


.69 


.71 


.72 


Average 


.70 


.73 


.74 


Form Perception 


Boys 


cn 

VO 

• 


.65 


.67 


Girls 


.62 


.65 


.67 


Average 


.62 


.65 


.67 


Clerical Perception 


Boys 


.66 


.70 


.73 


Girls 


.60 


.66 


.68 


Average 


.63 


.68 


.71 


Motor Coordination 


Boys 


.68 


.72 


.75 


Girls 


.72 


.76 


.82 


Average 


.70 


.74 


.79 


Finger Dexterity 


Boys 


.56 


.62 


.64 


Girls 


.58 


.62 


.67 


Average 

Manual Dexterity 


.57 


,62 


.65 


Boys 


.63 


.68 


.71 


Girls 


.65 


.69 


.74 


Average 


.64 


.69 


.73 



Aptitidis tf the 6ATB 

F « 4 Ctrrelelioii bilini iaitul Us! st .»« n 6"lts t. II, ul tl ul men worts it Sn<« 12. 
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TABLE 6 



DISTRIBUTION OF TETRACKORIC CORREIATIONS BETWEEN PASS-FAIL 
ON INITIAL SCORE AND PASS-FAIL ON RETEST SCORE FOR THE 3o CAP S 



Interval 



Grade 9 Sample Grade 10 Sample _ 

Boys Girls Total Boys Girls Total 



Grade 11 Sample 
Boys Girls Total 



90 - 


.94 


0 


0 


0 


85 - 


.89 


1 


0 


1 


80 - 


.84 


2 


5 


7 


75 - 


.79 


8 


8 


16 


70 - 


.74 


8 


6 


14 


65 - 


.69 


10 


12 


22 


>6o - 


.64 


7 


5 


12 



0 0 0 

2 2 4 

8 5 13 

7 13 20 

11 8 19 

6 6 12 

2 2 4 



112 
3 14 

8 8 16 

12 14 26 

10 6 16 

2 4 6 

0 2 2 



Man. 



.71 .71 .71 



•74- .76 .75 



.77 .77 .77 




wm 



tpawttttiil 
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sample and lowest for the ninth grade sample. The medians for the ninth, 
tenth and eleventh grade samples were .71, .75, and .77, respectively. 

Effect of Application of a Cutting Score Band on QAP Stability 

A further inspection of the graphs in Figure 5 show the considerable 
variation in stability of GAP's for each sample. For seme of the GAP's the 
stability is satisfactory even at a ninth grade level. On the other hand, 
other OAP's have unsatisfactory stability for use in counseling. This is 
particularly true at the ninth grade level where many of the GAP's have stability 
coefficients of less than .70. There is a serious question of whether GAP's 
with such low stability would be useful in counseling. Additional analyses 
of the data were made in an attempt to see whether it would be possible to 
iaprove GAP stability for a portion of the individuals. The objective of this 
analysis was to establish a ’’band" of cutting scores for each QAP which would 
identify individuals whose scores are close to the cutting scores for the GAP. 
These are the individuals for whom QAP instability is the greatest. The 
reason is that only slight changes in their test results are required for them 
to pass after failing initially, or to fail after passing initially. To the 
extent that establishing a score band is successful in identifying such 
individuals, it would be possible to increase stability when testing inter- 
pretation is limited to individuals who fall outside the band. 

The bands around GAP cutting scores were based on the standard errors 
of measurement for each aptitude as computed separately for the ninth, 
tenth, and eleventh grade samples. Thus, the width of the band around the 
cutting score for each aptitude in the norms for a particular GAP is equal to 
one standard error of measurement. This basis for the band width was chosen 
after some preliminary tryouts on small subsamples. 

The band widths are shown in Table 7. The differences were not great 
for the three sonnies but, where differences exist, the ninth grade band 
widths were highest and the eleventh grade band widths were lowest. The 
aptitudes with the greatest band widths were Finger Dexterity and Manual 
Dexterity, the aptitudes with the lowest stability coefficients. Table 8 
•hows the results of the application of the band in the case of QAP 1 for 
ninth grade boys. 

The three aptitudes in QAP 1 are Intelligence, Numerical Aptitude, and 
Spatial Aptitude. The four-way table for the total group shows that GAP 1 
had relatively high stability. (The tetrachoric correlation for this table 
was .82). Another, and perhaps more meaningful, wey of judging stability 
is on the basis of the proportion of incorrect predictions— individuals 
in the upper left and lower right cells of the four-way table. For the total 
group this proportion was .12, indicating that 12 per cent of those tested 
initially change from a pass to fail status or from a fail to a pass status 
when retested as twelfth graders. The four-way table for the within-band 
group shows that, for this portion of the total group, the proportion 
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TABUS 7 



WIDTH (+) OF BAND AROUND GAP CUTTING SCORES 



Aptitude 
G - Intelligence 
V • Verbal Aptitude 
N - Numerical Aptitude 
S - Spatial Aptitude 
P - Form Perception 
Q ~ Clerical Perception 
K - Motor Coordination 
F - Finger Dexterity 
M - Manual Dexterity 



Grade 9 

3 

3 

3 
5 

5 

4 

5 

6 

6 



Grade 10 
3 
3 

3 

4 

5 
4 
4 

6 
6 



Grade 11 
3 
3 

3 

4 

5 
4 

4 

6 

5 
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TABLE 8 



REIATIONSHIP BETWEEN PASS-FAIL ON INITIAL SCORE AND PASS-FAIL ON 
RETEST SCORES FOR OAP1 (ADULT NORMS G-125, M-115, S-115) FOR GRADE 9 BOYS 



Initial 
Test Scores 



Total Group 
Retest Scores 



0 

-ERIC* 





Fail 


Pass 


Total 


Pass 


7Wb) 


144(a) _ 


218 b+c * *12 


Fail 


2056(d) 


325(c) 


3l8o N 


Total 


2930 
Within ' 
Fail 


1+68 

’Band” 

Pass 


3398 

Total 


Pass 






91 b+c « .21 


Fail 


887~ 


197 


io§r n 


Total 


933 

Outside ' 
Fail 


242 1 

'Band” 

Pass 


1175 

Total 


Pats 


28 


99 


127 b+c = .07 


Fail 


1969 


327 


2090 N 


Total 1 


1997 
Total ( 
Fail 


22b 

Iroup 

Pass 


2223 

Total 


Pass 


28 
! 46 


99 1 

45 A 


218 


Fail 


• 887 
1969 


197 ! 

127 


3180 


Total 


2930 


468 


3398 



ubpi 
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misclassilied was .21, considerably higher than the .07 proportion i'c" the 
outside band four-way table shown next* 

This means that stability of OAP 1 can be improved if the counselor 
restricts his interpretation on OAP 1 to the individuals whose initial score^ 
are outside the band for this OAP, 

Similar results were obtained with other OAP * s • The medians an*., xanges 
of the proportions of incorrect predictions for the 36 QAP*s were as follov/s 
for the total group and for the group outside the band: 



Ninth grade 
Tenth grade 
Eleventh grade 



Range 


Total 


Outside 


Tots 


.06 - ".¥3 


.63 - .39 


.20 


.06 - ,42 


.03 - .39 


.19 


.05 - .43 


.03 - .39 


.10 



Median 



Outside 

.13 

.13 



DISCUSSION 



There are remarkably few inconsistencies and irregularities in the results. 
The similarities in practice effects for the three experimental samples '•.re 
striking,and the major findings apply equally to boys and girls. ;.’e find th, 
expected increases in mean scores attributable to maturation; there are no 
inconsistencies when results for the three experimental samples are compared; 
and the increases for boys and girls are -quite similar. The profiles ci 
stability coefficients are parallel, corresponding to parallel profiles o, 

•mean scores. Again, the boy-girl differences are quite small. It is reasonable 
to conclude that in terms of average effects of maturation and practice on 
GATB aptitude scores of high school students, we have about as good information 
as we can get. The results provide a good basis for making adjustments in 
aptitude scores (or aptitude norms) for effects of maturation or practice. 

But the problem of individual differences in rates of maturation or 
ability to benefit from practice remains. The results show clearly that 
these individual differences have an adverse effect on aptitude stability, 
with some individual aptitudes and some OAP’s having stability coefficients 
that are too low to be considered useful in counseling in lower high school 
grades. Additional research may provide a basis for increasing aptitude 
stability through additional testing or combining aptitude measures at the 
lower high school grade levels. Use of the band principle makes increased 
stability of measurement possible for 0AP*s when "he interpretation is 
restricted to individuals whose scores fall outside the band. 
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